Deep generative models (DGMs) are effective on learning multilayered representations of complex data and performing inference of input data by exploring the generative ability. However, little work has been done on examining or empowering the discriminative ability of DGMs on making accurate predictions. This paper presents max-margin deep generative models (mmDGMs), which explore the strongly discriminative principle of max-margin learning to improve the discriminative power of DGMs, while retaining the generative capability. We develop an efficient doubly stochastic subgradient algorithm for the piecewise linear objective. Empirical results on MNIST and SVHN datasets demonstrate that (1) max-margin learning can significantly improve the prediction performance of DGMs and meanwhile retain the generative ability; and (2) mmDGMs are competitive to the state-of-the-art fully discriminative networks by employing deep convolutional neural networks (CNNs) as both recognition and generative models.

Introduced in 2017, the Transformer is a deep learning model designed for NLP. Like recurrent neural networks (RNNs), Transformers handle sequential data. However, unlike RNNs, due to the attention mechanism, Transformers do not require that the data be processed in a sequential manner. This allows for much more parallelization in Transformers (in comparison to RNNs). In turn, parallelization during training allows for training on larger datasets.

Deep generative models (DGMs) are effective on learning multilayered representations of complex data and performing inference of input data by exploring the generative ability. However, little work has been done on examining or empowering the discriminative ability of DGMs on making accurate predictions. This paper presents max-margin deep generative models (mmDGMs), which explore the strongly discriminative principle of max-margin learning to improve the discriminative power of DGMs, while retaining the generative capability. We develop an efficient doubly stochastic subgradient algorithm for the piecewise linear objective. Empirical results on MNIST and SVHN datasets demonstrate that (1) max-margin learning can significantly improve the prediction performance of DGMs and meanwhile retain the generative ability; and (2) mmDGMs are competitive to the state-of-the-art fully discriminative networks by employing deep convolutional neural networks (CNNs) as both recognition and generative models. Papers published at the Neural Information Processing Systems Conference.

Generative models create data similiar to what they trained on Training these models is very hard One approach is using Generative Adversarial Networks (GANs) Facebook's Yann LeCun considers them "the most interesting idea in the last 10 years in ML" What are the differences between Discrimitive and Generative models? A discriminative model learns a function that maps the input data (x) to some desired output class label (y). In probabilistic terms, they directly learn the conditional distribution P(y x) A generative model tries to learn the joint probability of the input data and labels simultaneously, i.e. Facebook's Yann LeCun considers them "the most interesting idea in the last 10 years in ML" What are the differences between Discrimitive and Generative models? A discriminative model learns a function that maps the input data (x) to some desired output class label (y).